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ABSTRACT 

The effectiveness of a social skills training 
workshop was assessed by comparing the rated competence of 
participants in an Interpersonal Skills Training Program to the rated 
competence of nonparticipants • Subjects' self-ratings were included. 
This comparison was operat ionalized through a pretest-posttest design 
with 12 experimental and 22 control subjects. The assessment 
instruments consisted of Spitzberg and Hurt's (1987) Conversational 
Skills Ratify " 'le (CSRS) and Curran's (1982) Simulated Social 
Interaction Test (b\T). Two rating judges were utilized. Results, 
although n^d^st, 3' in the expected direction. Assessment of 
competence with the CSRS failed to reveal any significant improvement 
in the experimental group relative to controls. However, the SSIT did 
reveal significant improvement of the rated skills and anxiety of 
experimental subjects while the control group showed no significant 
improvement. In addition to assessing the effectiveness of the 
training program, this study sought to establish further validity 
support for the CSRS. As expected, the CSRS revealed a positive 
correlation to SSIT ratings. (Author) 
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Abstract 

Tha ef f activaneaa of a aociki akilla training workshop wag 
assessed by comparing tha ratad competence of participant! in an 
Interpersonal Skills Training Prograa to tha ratad competence of 
nonperticipants. Subjects 1 aelt-ratinga wara includad. This 
cospsrison waa opsrational izad tt augh s prstsg t-poi ttsit dsiign 
with 12 axpariaantal and 22 control subjecte. Ths aiftuntnt 
instruaants consisted of Spitzbsrg and Hurt's (1987) 
Convarsatlonal Skills Rating Scala (CSRS) and Currents (1982) 
Siaulatad Social Intaraction Test (SSIT). Two rating judgsi wara 
utilized. Results, although aodest, ara in tha axpactsd 
• dirsction. Asssssaant of coapatanca with tha CSRS failad to 
ravaal any significant iaprovaaant in ths axpariaantal 'group 
ralativa to control!. Howaver, tha SSIT did ravaal significant 
iaprovaaant of tha ratad skill and anxiaty of axpariaantal 
subjscts whils ths control., group showed no significant 
iaprovaaant. In addition to asseeaing tha sf fectivenesa o' ths 
training prograa, this study sought ts. ^blish furthsr validity 
support for tha CSRS. Aa axpactsd, ths CSRS rsvsalad a positive 
corrslation to SSIT rating*. 

© 

^ Paper prai.ntad at tha Speech Coeaunlcation Aeeocietlon 
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IMPROVING COMMUNICATIVE COMPETENCE: VALIDATION OF A 
SOCIAL SKILLS TRAINING WORKSHOP 



The plethora of training programs, measurement methodologies, and 
conceptualizations in the area of social and interpersonal skills can be 
traced to a core problem: the lack of a unifying, integrative framework. 
Clinicians have little basis for selecting, organizing and implementing 
training programs, given the enormous diversity and breadth of activities 
available. As yet, no single training approach has proven unequivocally to 
be superior empirically to alternative approaches (see Curran, 1977, 1979a, 
1979b; Hersen & Bellack, 1976). Furthermore, no single diagnostic 
technique has proven to be superior to other assessment techniques 
(Bellack, 1979, 1983; Eisler, 1976: Hersen & Bellack, 1977; Liberman, 1982; 

j Spitzberg, in press). These are not unrelated trends. Validation of 

I training methodologies presupposes valid measurement techniques to 

ascertain training effectiveness. Furthermore, until there are clear 
conceptual models of the nature of social skills and the learning process 
underlying these skill?), it will be difficult to develop measures that 
accurately reflect the skills and abilities being trained. The study 

i reported herein is directed toward these dual purposes: to examine the 

validity of an interpersonal skills measure, and to assess t w * 
effectiveness of an integrative model of interpersonal competence and a 

( training program designed from its perspective. 

; Although social skills training has been a topic for research for 

, decades, there has been little agreement on the most effective model for 

the design and implementation of training programs. Different grounding 
theories of learning lead . very different types of sWills training 
programs (Ellis 4 Whittlngton, 1981). 
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SOCIAL SKILLS VALIDATION 2 
Recently , training programs with eclectic learning approaches have 
increased in popularity (KurU, Marshall A Banspach, 1985). These social 
skills training programs are combining role-playing, modeling, cognitive 
restructuring, and behavioral conditioning with successful results, at 
least for short-term changes in behavior (Ellis A Whittington, 1981), 
However, while the eclecticism in social skills training shows promise for 
the successful tranafer of learning, there is a distressing lack of 
conceptuil coherence upon which to build and validate social skills 
training programs. 

Social skills training programs have generally focused on social 
skills deficits; that is, ertinguishing ineffective behaviors. Some 
training programs nave addressed shyness or heterosexual-social anxiety 
(see Curran, 1977). Other training programs have achieved wide popularity 
in a specific remedial area, e.g., assertiveness training (see Galassi A 
Galassi, 1976). Still other training has focused on improving psychiatric 
prtients 1 and mentally retarded individuals* abilities to function more 
effectively in society and tc be more self-reliant (e.g., learning to make 
requests and to ask questions). 

Regardless of the success of the various social skills training 
programs, the concept of social skills is rarely given adequate attention 
in research studied. Spitzberg and Cupach (198U) offer a comprehensive 

# 

conceptualization of social skills which they model in terms of relational 
competence. Because competence is a quality perceivel by others (McPall, 
1982), it is a relational concept: 

Relational competence can be defined conceptually as the extent to 
which objectives functionally related to communication are fulfilled 
through cooperative interaction appropriate to the interpersonal 
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SOCIAL SKILLS VALIDATION 3 
context. Therefore, relationally competent communication _ w 
conceptualized as a function of perceived appropriateness and 
effectiveness (Spitzberg A Cupach, 198*1, p. 100). 

This -vuel of relational competence offers a potential soluvion to the 
lack of conceptual coherence in the research studies of social skills 
training. Their model of relational competence is organized around three 
components: motivation, knowledge, and skill. If eclectic learning 
activities can be organized according to the three components of rel?tional 
competence and interpersonal skills are positively affected, a major step 
will have been taken in validating a unifying framework for social skills 
training. 

The effectiveness of training programs based on a comprehensive model 
of communication competence has not been tested. This study tests the 
effectiveness of a serial skills workshop, or interpersonal skills training 
program (ISTP), based on a comprehensive model of relational competence. 
This purpose is achieved by examining the effects of the craining program 
on tl.e ratings of participants 1 interpersonal competence as determined by a 
pre- and postassessment, compared to a control group of training program 
nonparticipants, and to self-reports. 

Unlike many training techniques, the ISTP takes a more comprehensive 
vie* of the nature of interpersonal competence. For example, many training 
programs view social skills problems primarily as motivational deficits 
(e.g., rhetoritherapy, systematic desensitization, cognitive restructuring) 
or skills deficits (e.g., heterosocial skills, assertiveness skills). The 
relational competence model assumes that communication difficulties may 
have multiple etiologiea. Persons may experience communication failure 
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♦ .„ innovation in training content per m. but a new 
ISTP does not represent an innovation » 

„r Misting eclectic training techniques 
w , y of organizing and making sense of existing 

under the aegis of a single Prograa. 

The Pri-ar, thrust of this stud, is to e^nlne the efficacy of - 

i,sue. the vaUdl* of the assures used for assessment -ust he addressed. 
lf the Conversational Skills Bating Scale (CSRS, «rf Sifted Social 
interaction Test (SSIT) are valid measures of interpersonal skill, they 
ohould he positively correlated to each other. 
HYPOTHESIS It 

r» Conversational Skills Bating Scale (CSBS) is positively related to 
the Simul.tei Social Interaction Test (SSIT) . 
P^ing that there Is validity for the CSBS and SSIT. the 
notation folio- that sublets receiving training in interpersonal 
sklll s «U shov significant and positive isprovcent relative to suhjects 
receiving no training. 
HYPOTHESIS 2: 

Participants in an interpersonal skill, training progra. -Ill 3hov 
slgnlfl^t positive in.prove.ent in their «Ud interpersonal skill,. 
whU , a control group of subjects not participating in the training 
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SOCIAL SKILLS VALIDATION 5 
program will show no significant change in their rated interpersonal 
skills. 
Method 

At a large southwestern public university, an Interpersonal Skills 
Training Program (ISTP) has been developed based on Spitzber^ and Cupach's 
model of communication competence. It is conducted each semester for 
Swudcnti <ho receive low ratings on their in-class interpersonal competence 
activity by their communication instructor. One of those training programs 
provided an opportunity to test the effectiveness of the ISTP, and by 
implication, its eclectic training model on the participants 1 resultant 
rated competence. 

In order to determine what improvement, if any, such a training 
program could facilitate, a pretest and posttest design with an 
experimental group of participants and a control group of nonpar tici pants 
was chosen. Such a design would provide relatively unambiguous results 
with clear comparisons of posttest ratings of competence to the pretest 
ratings, as well as a comparison of the experimental group to a control 
group. Isolation of observed differences Would therefore be enhanced. 
Selection of subjects 

Study subjects were recruited from 38 sections of the basic 
communication course. Students who qualified for training program 
participation but could not attend served as control subjects. 

AH training program participants (experimental group) and 
nontreatment volunteers (control group) were videotaped in dyadic 
conversations with each other and in simulated role-plays individually. 
Videotaping of subjects was done the week preceding and the second week 



7 



SOCIAL SKILLS VALIDATION 6 
roll*,!* the training progr«i a period of three week, separated the 
^Uet session and the posttest swsie . Conversation pair, consisted of 
» wriety of experimental and control subject combinations: 
WTi-nUl/control, experimental/experimental, and control/control. 
Hatching was based entirely upon the sub; signing v.p for Pretest and 
po.tt.st time slots consistent with their individuU schedules. 

' A ft er Ul subjects were videotaped chronologically, rating tapes were 
record* to 1* the Pretest and posttest conditions, consolidating 
conversation, for CSRS rati** purposes and consolidating role-plays for 
SSIT rating purposes. 
Assessment Instruments 

tn his analysis of social skills and interpersonal competence 
m„,ur.s, Spitzberg 0986.. 1986b) has identified as many as 138 relevant 
.Msures. However, despite the number of measures, Spitzberg concludes 
that few of them have been extensively researched. Such diversity of 

.... , .... * tt.Mw.rn <, a i m for a flezible, convenl jnt, 
assessments makes it clear that there is a i to tor ■ 

and validated measure of skills. For the assessment of the social skills 

or co«niction competence of videotaped subjects, the ideal measurement 

should be simple and uncomplicated yet valid and reliable enough to assess 

communication competence accurately. The Conversational Skills Rating 

Soale (CSRS) (Spitzberg h Hurt, 1987) was developed to ceet this need. 

CSRS . Developed initially through -Hot studies and literature 

March, molecular-level, discrete behaviors "-hat can be observed and rated 

(Spitzberg, 1985) comprise 25 of the 30 items oa the CSRS. The skill items 

identified on the CSRS include such behaviors as -use of eye contact;" 
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SOCIAL SKILLS VALIDATION 7 
"vocal volume (neither too loud or too soft);" "asking of questions." The 
remaining five items are oolar-level evaluations of overall conversational 
skill, expressiveness, altercentrism, composure, and 
appropriateness/effectiveness. The 25 molecular items are rated on a 5- 
point Likert-type scale, anchored as fellows: 1 s INADEQUATE (use was 
awkward, disruptive, or resulted in a negative Impression of communicative 
skill?); 3 - ADEQUATE (use was sufficient but neither very noticeable nor 
excellent. Produced neither positive nor negative impression); 5 s 
EXCELLENT (use was smooth, controlled, and resulted in positive impression 
of communicative skills). 

In aodition to providing diagnostically useful assessment of 25 
molecular conversation behaviors and 5 molar competence ratings, the CSRS 
is self-explanatory and easy to use by untrained raters. 

Spitzberg and Hurt (1987) assessed the CSRS both as a self-report 
measure and as an observational measure of in-class student "get- 
acquainted" conversations. The results of that study supported the 
reliability, convenience and powerful relationship of the behavioral item* 
to the molar ratings of interactants' communicative performance. 

SSIT. In addition to ratings of 25 molecular behaviors provided by 
the CSRS, another measure was desired to serve as a comparison for the 
CSRS. Curran's (1982) Simulated Social Interaction Test (SSIT), a proven 
role-play instrument, was chosen because of its empirical validity and for 
its ease of administering. Careful training can ensure high interracer 
reliability for Judges. 

The SSIT is composed of a role-play orientation narrative and a script 
of eight problematic situations that are presented to individual subjects 
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SOCIAL SKILLS VALIDATION 8 
who respond verbally in role-play fashion. The eight role-play situations 
consist of three parts: 1) the narrator describing the situation; 2) a 
confederate (or narrator) delivers ng the situational prompt; and 3) the 
aubject's response to the prompt. The eight situations address situations 
many people have trouble with, e.g., dealing with disapproval or criticism, 
expressing interpersonal warmth, and receiving complements. For example, 
the situation dealing with disapproval or criticism is set up for the 
respondent as follows: 

NARRATOR: You are a work, and one of your bosses has just finished 

inspecting one of the joos that you have completed. He says to you: 

CONFEDERATE: "That's a pretty sloppy job. I think you could have done 
better.* 

The subject then responds to the confederate's remark as if he or she 
were actually in that situation, e.g., "I'm sorry. In what way did I 
Raters make molar evaluations on an 11-point Likert-type scale of the 
subjects response to each situation on two dimensions: social 
skillfulness and anxiety. 

The SSIT has been tested extensively (see Curran, 1982) and, compared 
to other measures of competence, the SSIT is one of the beat validated 
measures available (Spitzberg, 1986b) 
Ratsrs 

Because time requirements for rating subjects would be demanding (20 
hours), there were two volunteers from the cadr of graduate students in 
communication. If interrater reliability were controlled, two raters could 
provide the necessary ratings, because the volunteer raters were graduate 
assistant instructors in the basic cor aunication course, they were versed 
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SOCIAL SKILLS VALIDATION 9 
in using the CSRS instrument in rating students in their classes; training 
was thus streamlined and compressed to address the SSIT alone. 

One training session was conducted that 1) familiarized the raters 
with the SSIT role-play situations, 2) provided criterion behaviors as 
behavioral observation .-chors, 3) utilized videotaped sample SSIT role- 
plays as practice, and *l) included discussion and comparison of rater 
practice ratings. After the first round of practice ratings, criterion 
behaviors were again discussed in order to identify perceptual 
discrepancies and to anchor raters' perceptions of subjects* skillfulness 
and anxiety behavior. 

Two more rounds of practice ratings followed by discussion of 
behavioral anchors ana comparison of ratings were conducted; rater 
agreement improved with each round, the third round producing acceptable 
agreement levels. Percentage of near-agreement of the SSIT items on the 
first two p actice ratings averaged about 63%, improving to 81* agreement 
on the third round. Considering the similarity of the two raters' graduate 
and teaching experiences, and the positive outcome of the intensive 
training session, rater agreement ranging between 70* and 80* on the 
ratings of the study subjects was expected. 

Results • 

The average construct scores are examined first; that is, the ratings 
by the two raters on the major variables are summed and divided by two to 
provide a mean construct rating. In cases where these findings do not meet 
with expectations, or are anomalous, individual rater analyses are 
performed. It is hoped that by examining the individual raters in such 
cases the reasons for the anomalous results can be elucidated. 
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Bel lability 

Interrater reliability was „„., e d by Pearson cor- . j between 
rater,' scores , ., tne CSRS, SSIT/,kill and SSIT/anxiety construct,. 
Coefficients, shown in T.ble 1, , re low in spite of preUa«.nary rater 
training ard lndicatior ^ prior resear-h that the SSIT sh.u.'cl have 
achieved high Joefficient, (Curran, 1982). 



TABLE 1 ABOtn HERE 



Desoite these ^scorn-aging results, there are several reasons to 
continue analysis. rir3t , pu bli3nea r „ esrch h „ reported ^ 
Pliabilities for Active rating instrument, (e.g., Waltz A Cough. 
198*). second, for exploratory purpose,, results should still be examined 
to determine if there i, reason for further research. Third, to the extent 
that significant results are found despite rater disparities, we can 
conclude that the constructs studied are powerful enough to overcome these 
statistical problems. And last, given that validity i, ultimately a more 
1-PorUnt question thu, reliability, experts h .ve recommended using 
averaged scores to enhance both the reliability a„d validity of rating, 
(Strahan, 1980; Horowitz. l noU ye t Siegelman. 1979). 

Internal consistency of the measure w„ ,„„,„, by the coefficient 
iph. reliability. Si . l0 e thi, statistic i, a function of M Jix ., th .' 
coefficient produced can be con,id.red an extremely conservative e,tlmate 
of internal reliability. The CSRS produce, coefficient, for the Pretest 
and ,o,tt.,t condition, for Rater 1 of .75 and .85, re,p,ctively. I, 
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SOCIAL SKILLS VALID ATI OK 1 1 
higher coefficient for the posttest condition suggests a possible learning 
effect in using the SSIT. 

Examination of the coefficient alpha for the SSIT is sore complicated. 
In addition to having a small sample broken down on the aubsca' s, the SSIT 
constructs of skill and anxiety consist of only eight items each. Since 
coefficient alpha is a function of the number of items and sample size, the 
coefficients produced are certainly deflated. Table 2 displays the 
coefficient alpha for the SSIT skill and anxiety constructs broken down by 
rater and condition. Interestingly % opposite learning effects seem to have 
occurred by the raters. Rater 1 appears to have become less reliable 
whereas Rater 2 became ^ore reliable. 

TABLE 2 ABOUT HERE 



Validity 

Support was found for Hypothesis 1; the CSRS instrument correlated 
positively to the SSIT instrument (rt.55, £< .01). It is interesting to 
note that i in addition to the extensive research literatures supporting 
both the CSRS and SSIT, the averaged CSRS competence protest ratings were 
significantly related to SSIT/skill posttest ratings three weeks later 
(r=.55 f £<,001) and to SSIT/anxiet) *test ratings (r=,67, £< .001). 
This rrovides evidence of the utility oi" the CSRS instrument. 

As would be expected, SSIT/skill pretest ratings are significantly 
related to SSIT/anxiety pretest ratings (£=.7*, £<*001), SSIT/skill 
posttest ratings 5, fc^.01), and SSIT/anxiety posttest ratings (r=.H8, 
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SOCIAL SKILLS VALIDATION 12 
£<.OOl). CSRS competence pretest ratings are substantially related to 
CSRS competence posttest ratings (rs.78, £< .001). 

In short, the constructs appear to be relating to one another in ways 
that would be expected and, in s' je cases, reveal impressive power for such 
a small sample. 
Test of the Workshop 

The essential purpose of this study was to assess th-» effectiveness of 
an interpersonal skills training program (ISTP). To determine this, three 
constructs were used as dependent variables: CSRS ratings, SSIT/skill 
ratings and SSIT/anxiety ratings. As discussed earlier, these constructs 
are averaged across raters except in instances where results are 
counterintuitive. The expectation for these constructs was that each would 
show significant increases in the experimental (ISTP) condition but not in 
the control condition. 



TABLE 3 ABOUT HERE 



Hypothesis 2 was partially supported. CSRS ratings shown in Table 3, 
did not reveal a significant change in the experimental or control 
conditions. The female experimental subjects show sr.me improvement over 
the controls, while male experimental subjects do not. Figures 1 and 2 
illustrate this disparity. ■ . ' 



FIGURE 1 ABOUT HERE 
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SOCIAL SKILLS VALIDATION 13 



FIGURE 2 ABOUT HERE 



In order to assess the reason for observing no significant change in 
the experimental condition, an analysis of rater differences in the CSRS 
was performed (Table H). Raters 1 and 2 varied significantly in their 
ratings of the experimental group. Rater 1 found no significant difference 
in those subjects 1 behavior in a pretest /posttest comparison. On the other 
hand, Rater 2 did indeed perceive significant improvement in the 
experimen'al subjects* behavior in the CSRS pretest/posttest comparison. 
In short, it appears that Rater 1 perceived no such change, as assessed by 
the CSRS instrument. A possible reason for this is addressed in the 
discussion below. 



TABLE 4 ABOUT HERE 



In the examination of the SSIT/skill ratings, results (shown in Table 
5) were found that supported the efficacy of the skill training program. 
Subjects were rated as significantly more skillful in the experimental 
condition while no change was observed in the control condition. 



TABLE 5 ABOUT HERE 



The SSlT/snxiety ratings in Table 6 reveal a similar pattern to the 
SSIT/skill ratings. Subjects in the experimental condition were perceived 
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SOCIAL SKILLS VALIDATION 1*1 
u 3lgnificantly less anxious after the training program, while no change 
was observed in the control subjects. 



TABLE 6 ABOUT HERE 

Analysis of variance using the SSXT/skill posttest ratings as the 
dev«ndent variable crossed by condition and sex reveals an interaction 
effect. While condition prouuced a nonsignificant main effect and sex a 
significant cm effect, the variables reveal a significant interaction 
effect that explains almost 22* of the variance (r».H6) (see Table 7). 
Figures 3 and H illustrate comparisons of SSIT/skill mean ratings by 
treatment condition and sex of subjects. 



TABLE 7 ABO 07 HERE 



PIG ORE 3 ABOUT HERE 



FIGURE * ABOUT HERB 

Using the ?3IT/anxxety posttest ratings as the dependent variable 
crossed by condition and sex, analysis of variance reveals significant main 
•ffHt. and nonsignificant effect, for each of the variable, in iaol ation 
and interaction. The overall TO del approaches significance and explains a 
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SOCIAL SKILLS VALIDATION 15 
substantial amount of variance (r=.21) (see Table 8). Figures 5 and 6 
illustrate comparisons of SSIT/anxiety mean ratings by treatment condition 
and sex of subjects. 

TABLE 8 ABOUT HERE 



FIGURE 5 ABOUT HERE 



FIGURE 6 ABOUT HERE 
Discussion 

The purpose of tr±s study was to determine if an interpersonal skills 
training program (ISTP) would facilitate significant improvement in 
participants' interpersonal skills or competence. For the most part, the 
results have * »ved that this training program has indeed shown 
effectiveness, despite some reliability problems. 
Rater Differences 

Although reliability coefficients were far below the .70 level 
anticipated for this study, there is encouragement that these low 
coefficients can be improved substantia: -Y with more rigorous rater 
training. 

J-, contrast to the rater training in this study, Curran (1982) trained 
lay people as well as communication specialists were utilized as rater 
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trainee, In a ver, lengthy and thorough procedure that included extenaive 
.xplanation of oriterion behavior, rationale and 12 hour, of practic. 
rating, by rater trainee,. Oniy trainee, with a degree of agreement with 
the criterion rating, of .80 or better -ere selaeted to rate the 
exp.ri.ental tapes. The,, training procedure,, prootored by a re,e.roh 
tM.. lead to high interrater reliability and agreement with criterion 
ratings* 

Onfortunately, such a pool of volunteer (or paid) rater, -a, not 
practical for thi, ,tudy due to the extreme time demand, of such 
participation on graduate ,tudent volunteer rater, and to budgetary 
limitation,. With the,e con,traint,. one 5-hour training ,e„ion wa, 
designed to anchor behavioral criteria for the SSIT rating,, and interrater 
reliability v., =lo,e to .70 at the end of that training ,e„ion. 

Becauae both volunteer rater, -ere al,o teaching a„i,tant, in the 
communication department, they were already familiar with the CSRS r.txng 
inetrum.nt. Extensive rater training of the CSRS ha, been an integral 
component of teaching a„i,tant,. training each fall. Previous etudie, 
T 1id.ting the CSRS Hatrument (Spitzberg A Hurt. 1987; Spitzberg, 1985; 
Spitzberg. 1986.; Spitzberg. 1986b) have found generally high interrater 
reliability, localizing rater difference,. 

D«,pite reting, of significant improvement in the experimental group 
a. compared to the control group on the SSIT by both rater,, the CSRS 
rating, proved contradictor to expectation,: Rater 1 observed no 
•ignlficant improvement in either experimental or control group aubject,. 
while Rater 2 did ob.erve ,ig,'.f lent improvement in the experimental group 
but not in the control group. 
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SOCIAL SKILLS VALIDATION 17 
Several factors could be contributing to such disparity. One factor 
that stands out is that the two raters were perceiving subjects' behavior 
differently. In fact, some items reveal an inverse relationship: the 
higher one rater scored the behavior, the lower the other rater scored it. 
This phenomenon occurred on only a few, isolated items, but it is 
perplexing nonetheless, considering the raters' common teaching experience. 

Another factor that could have contributed to the rater differences is 
that, although both raters were peer teaching assistants, the raters 
participated in different teaching assistant training programs (one year 
apart). The training methods and criterion behaviors utilized in the two 
different sessions to anchor ratings on the CSRS could very well have been 
substantially different to cause lower coefficients. It is even possible 
for each rater to be highly correlated with his own training group, yet not 
significantly correlated with another teaching assistant from a different 
training group. It would be interesting to see how well individuals with 
no specialized knowledge in interpersonal communication might correlate 
with one another with training similar to that in this st"<ly. Curran 
(1982) had success with such raters. 

Finally, the measures used in this study are subjectively scaled 
instruments, and the criteria for Judging competence are inherently 
subjective. Consequently, high reliability is not expected. Therefore, 
averaged scores are more consensual in the present study. 
Subjects' Self-Ratinfls 

The subjects self-ratings were not correlated with any other ratings 
except other self-ratings. The CSRS posttest self-ratings were predicted 
by the pretest self-ratings; the subjects were rating themselves 
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consistently over tine* The more a subject perceived and rated improvement 
on the SSIT/skill dimension between pretest and posttest, the higher those 
subjects* CSRS self-ratings were. 
Effectiveness of the Training Program 

Despite the problems with interrater reliability, the results show 
improvement of the experimental group subjects in comparison to the control 
group subjects Although the overall CSRS competence ratings indicate no 
significant change in the experimental subjects, means are in the expected 
direction. The SSn/skill and SSIT/anxiety ratings are stronger in showing 
significant change in the experimental group over the control group. 

The results show movement in the expected direction and indicate the 
potency of the training program to facilitate improvement in participants 1 
rated competence and social skills. By tightening up the rater training 
component and examining students who are in greater need in training (i.e., 
students with lower oaseline skill levels), it seems realistic to expect 
that tin limited improvement of experimental subjects' ratings for this 
study points to a stronger showing of improvement in future studies of this 
type. 

An interesting result of the analysis of the SSIT/skill and 
SSIT/anxiety ratings reveals an interaction effeot between males in the 
experimental condition and males in the control condition. There was 
generally some increase in ratings of the posttest over pretest ratings' for 
both groups of females and for experimental males; yet, control group 
males' rated skill and anxiety decreased. An explanation could be that the 
oontrjl group males perceived the exercise as a negative one, their only 
reward for participating in the study the receiving of class extra-credit 
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points. However, due to the small sample size of the experimental-group 

males (n=*D , this result must be »idered cautiously. 

Instrument Validation 

Disappointingly, the CSRS instrument revealed a lack of significant 

j results in measuring the experimental group's improvement after attending 

the training program. On the CSRS, the raters were perceiving different 

things in the subjects' behavior. Nevertheless, the CSRS, like all other 
i 

observation measures, is based on raters 1 subjective evaluations, and 
raters apply different standards because they are individuals. However, 

» the beauty of the CSRS assessment is that it fills an assessment gap \ty 

providing specific information that can be diagnosed in almost any 

■ interpersonal context; it can be used as a self- or other- reference; and 

it can be used by trained or untrained raters. The caveat, however, is 
that the procedure* for using it must be defined critically. In fact, 
Spitzberg (1986a) warns n if 'objective' information is desired, rater 
training and further scaling refinement is likely to be necessary for the 
raters to provide consistent information." 

Regardless of its lack of significant results, the CSRS pretest scores 
did show a positive correlation with the SSIT/skill posttest scores — 
ratings made three weeks later — as well as with the SSIT/anxiety pretest 
ratings. This correlation is important because it reveals that the CSRS is 
doing what it was intended to do: identifying competent behavior. As it 
should be, the CSRS is also positively correlated to itself, pretest to 
posttest. Heeding Spitzberg »s caveat about rater training could indeed 
enable CSRS assessment to be a powerful instrument in measuring behavior 
comparisons such as this study. 
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Implications 

Methodologically speaking, ftp study design, .ike this one to provide 
sensitive results, -ore thorough and highly struotured rater training 
*ould b. emphasized. Nor. raters are ,1 S0 needed to facilitate higher 
reliability of the assessment instruments. If the time element is 
constrained (as it was for this study), using only o.,e measure is 
recommended. 

The training Program schedule itself could be modified. As presented, 
the training program took place in two 6-hour session, three weeks apart. 
Dissipation of results of the first session could have occurred in the 
Interim. 

The Interpersonal Skills Training Program could easily fill the time 
requirecents for an entire 3-hour university course; perhaps more solid 
results could be obtained in that w,y. The training program as it was 
conducted in this stud, was designed and produced by graduate students in 
fttlfill«.nt of a course assignment. Up to now and including the training 
pre* am that was measured in this study, there has been no real standard or 
continuity of design for presenting the training program since a different 
group of graduate students individualized it each semester (within the 
fra.»work of the course guidelines). How that the ISTP has been approved 
by the university, the training program's standard protocol should solidify 
and become more consistent. Put-u-e studies such as this one could possibly 
find stronger and mure positive results for program participants. 

Summary 

This study shows mixed results by instrument and by rater. However, 
taken as a whole, the data suggest considerable improvement for training 
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program participants' rated competence. The Interpersonal Skills Training 
Program offers significant potential for the enhc-^ement of students' 
skills, cs deduced from the improvement of training program participants* 
skill ratings on the Simulated Social Interaction Test and, according to 
one rater's scores, the Conversational Skills Rating Scale. 

It is our opinion that better rater training and stricter selection 
criteria for subject participation (i.e., low competence levels) w< aid 
facilitate significant, uniform improvement of participants' rated 
interpersonal skills. 
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Tabic 1 



Interrater Correlation Coefficient Assessed 
by Simple Pearson Correlation 



Variable 



Coe. .icient 



Significance 



CSRS 

Pretest 
Posttest 

SSIT/Skill 
Pretest 
Posttest 

SSIT/ Anxiety 
Pretest 
Posttest 



.53 
.54 



,53 
.56 



.53 
.58 



.01 
.01 



.001 
.001 



.001 
.001 



Table 2 

Coefficient Alpha for the SSIT 
by Rater and Time 



SSIT Dimension 

Skill 

Pretest 
Posttest 



Rater 1 



.79 
.68 



Rater 2 



.67 
.81 



Anxiety 

Pretest 
Posttest 



.79 
.73 



.65 
.82 
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Table 3 



Correlated T-Tests Comparing Pre- and Posttest 
Cr 'Defence Ratings in the Experimental 
and Control Groups 





Experimental 


Control 




Pre 


Post 


Pre 


Post 


Sample Means 


51 .69 


53.88 


50.13 


50.05 


S.D. 


3.62 


5.59 


5.33 


6.64 


S.E. 


1 .28 


1.98 


1 .38 


1 .71 


t Value 




-1.57 




0.08 


£ 




.16 




.94 


n 




8 




15 



Table 4 

Correlated T-Tests Comparing Pre- and Posttest 
Competence Ratings by Raters 



Rater 1 Rater 2 

Pre Post Pre Post 

Mear: 50.38 51 .38 51 .80 56.60 

S.D. 3.19 5.66 6.61 7.39 

S.E. 1.13 1.20 2.09 2.34 

t Value -0.41 -3.52 

2 .689 .006 

n 8 10 
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Fig. 1 — A comparison of CSRS mean ratings 
of female subjects. 
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Fig. 2 — A comparison of CSRS mean ratings 
of male subjects. 
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Table 5 



Correlated T-Tests Comparing Pre- and Posttest 
3SIT Skill Ratings in the Experimental 
and Control Groups 



Experimental Control 

Pre Post Pre Por t 

Sample Means 42.59 54.41 47.05 47.68 

S.D. 10.09 7.52 8.92 11.46 

S.E. 3.04 2.27 1.99 2.56 

t Value -3.93 -0.32 

£ .003 .752 

n 11 20 



Table 6 

Correlated T-Tests Comparing Pre- and Posttest 
SSIT Anxiety Ratings in the Experimental 
and Control Groups 



Experimental Control 

Pre Post Pre Post 

Sample Means 44.09 52.55 45.18 45.97 

S.D. 9.01 4.66 7.99 9.87 

S.E. 2.72 1.40 1.83 2.26 

t Value -3.49 -0.54 

£ .006 .595 

n 11 19 
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Table 7 



Analysis of Variance of SSIT Skill 
Posttest Ratings Crossed by 
Condition and Sex 





Sum of 




Mean 








Sauares 


df 


Square 


F 


Sig. 




763.334 


2 


381 .667 


4.52 


.02 


Main Effects 


196.667 


1 


196.667 


2.33 


ns 


Condition 
Sex 


375.456 


1 


375.456 


4.45 


.04 


Interactions 


345.455 


1 


345.455 


4.10 


.05 


Explained 


1108.790 


3 


369.597 


4.38 


.01 



r 2 » .22 
r « .46 



Table 8 

Analysis of Variance off SSIT Anxiety 
Posttest Ratings Crossed by 
Condition and Sex 





Sura of 




Mean 








Squares 


df 


Square 


F 


Siq. 


Main Effects 


503.641 


2 


251.820 


3.75 


.04 


Condition 


190.924 


1 


190.924 


2.84 


ns 


Sex 


166.779 


1 


166.779 


2.48 


ns 


Interactions 


39.431 


1 


39.431 


0.59 


ns' 


Explained 


543.071 


3 


181 .024 


2.70 


ns 



r 2 » .21 
r * .46 
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Fig. 3 — A comparison of SSIT skill 
mean ratings of female subjects. 
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Fig. 4 — A comparison of SSIT skill 
nean ratings of male subjects. 
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Fig. 5— A comparison of SSIT anxiety 
mean ratings of female subjects. 
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Fig. 6 — A comparison of SSIT anxiety 
mean ratings of male subjects. 
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